1. Technical Field
The present teaching relates to methods, systems, and programming for distributed computing. Particularly, the present teaching is directed to methods, systems, and programming for distributed application stack test certification.
2. Discussion of Technical Background
Distributed computing is a field of computer science that studies distributed systems, which include multiple autonomous computers or parallel virtual machines that communicate through a computer network, such as a computer cluster having multiple nodes. The machines in a distributed system interact with each other in order to achieve a common goal. A computer program that runs in the distributed system is called a distributed application. In distributed computing, a problem is divided into many tasks, each of which is solved by one or more computers, such as the nodes of a computer cluster. Distributed systems and applications may be applied as various paradigms, including grid computing, utility computing, edge computing, and cloud computing by which users may access the server resources using a computer, netbook, tablet, smart phone, game console, set-top box, or other device through the Internet.
For instance, APACHE HADOOP is a software framework that supports data-intensive distributed applications under a free license. It enables applications to work with thousands of nodes and petabytes of data. Rather than rely on hardware to deliver high-availability, HADOOP is designed to detect and handle failures at the application layer, thereby delivering a highly-available service. HADOOP is deployed on a computer cluster in the form of a HADOOP stack, which includes a set of software artifacts (HADOOP components), such as HADOOP software, configuration files, libraries, links, source code, documentations, miscellaneous, etc. The deployment of HADOOP on a cluster of machines usually involves hardware installation, operating system installation, update, and configuration, JAVA installation and configuration, and HADOOP stack installation, configuration, and diagnostic.
A distributed application stack may be configured in different ways by different users and deployed on different computer clusters. For example, a HADOOP stack may contain different software artifacts and each software artifact may have different versions to constitute different HADOOP configurations. Therefore, it is desired to certify a particular configuration to be deployed on a certain cluster in order to indicate whether the HADOOP stack in that configuration would work properly once deployed. However, currently, there is no easy way to ascertain that a particular set of versions comprising given HADOOP Stack deployment has been tested in a particular configuration for various reasons. For instance, not all HADOOP Stack artifacts are version controlled and recorded during and after the deployment; the hardware configuration on which a HADOOP Stack is deployed is not recorded; there is no record being kept of artifact version changes during certification or different versions of HADOOP Stack component being tested together.
Therefore, there is a need to provide a solution for automatically certifying a distributed application stack, such as the HADOOP stack, in a particular configuration, to be deployed on a certain cluster, such that the deployment and test results are fully tracked and recorded in a certification record, and the certification records are stored and published.